It will be appreciated that computers or indeed any data processing device such as a camera, if left unsecured, can grant access to a user's private information, for example: email correspondence, address book, photographs, videos, any unsecured documents or potentially bank account details.
Biometric recognition refers to automatic identification of individuals using their physiological and/or behavioural characteristics. Biometric recognition can ensure that private or sensitive information is accessed only by a legitimate user and by nobody else. While biometric recognition systems may get extremely accurate even for large scale identification applications, usually their performance comes with the cost of a sense of intrusion which leads to low acceptability from users.
Jain, Anil, Hong, Lin, and Pankanti, Sharath, “Biometric identification”, Communications of the ACM, 43 (2): 90-98, 2000 discloses the following bases for biometric recognition: DNA, ear (shape and structure of the cartilaginous tissue), face, facial, hand and hand vein infrared thermograms, fingerprint, gait, hand and finger geometry, iris, keystroke, odor, palm print, retinal scan, signature and voice. In addition, Unar, J. A., Seng, Woo Chaw, and Abbasi, Almas, “A review of biometric technology along with trends and prospects”, Pattern Recognition, 47: 2673-2688, 2014 have added finger knuckle print and tongue print. Little, James and Boyd, Jeffrey, “Recognizing people by their gait: the shape of motion”, Videre: Journal of Computer Vision Research, 1 (2): 1-32, 1998; and Nixon, Mark S and Carter, John N., “Automatic recognition by gait”, Proceedings of the IEEE, 94 (11): 2013-2024, 2006 disclose using gait as a basis for biometric recognition.
Recently, human hand tremor has also been identified as a potential basis for biometric recognition. Human tremor can be categorized in two main classes: resting tremor (which can be noticed when the muscles are not contracted) and action tremor. Action tremor manifests during a voluntary muscle contraction and encompasses postural, kinetic, intentional, task-specific and isometric tremor. This is the type of tremor that is the most likely to appear when using a smartphone.
Veluvolu, Kalyana C and Mg, Wei Tech, “Estimation of physiological tremor from accelerometers for real-time applications”, Sensors 11.3 (2011): 3020-3036 discloses a method to separate tremor (both voluntary and involuntary) data from noise in a raw sensor signal.
U.S. Pat. No. 7,773,118 discloses that involuntary tremor can be efficiently distinguished from voluntary movement or an action tremor.
Natalia Neverova, Christian Wolf, Griffin Lacey, Lex Fridman, Deepak Chandra, Brandon Barbello, Graham Taylor, “Learning Human Identity from Motion Patterns”, arXiv:1511.03908 discloses using deep networks to determine the relevant features for identifying a person. The obtained biometric is a combination of gait, tremor and gesture.
U.S. Pat. Nos. 8,994,657 and 7,236,156 disclose a handheld device, such as a free space pointing device, which uses hand tremor as an input. One or more sensors within the handheld device detect a user's hand tremor and identify the user based on the detected tremor.
U.S. Pat. No. 8,180,208 discloses identifying a photographer by determining a steadiness signal related to movement of a camera held while capturing an image or video. The steadiness signal is compared to a set of steadiness signals to produce a set of similarity scores between the photographer holding the camera and known photographers; and the set of similarity scores is used to identify the photographer that has captured the image or video.
Mansur, Paulo Henrique G, Cury, Lacordaire Kemel P, Andrade, Adriano O, Pereira, Adriano A, Miotto, Guilherme Alessandri A, Soares, Alcimar B, and Naves, Eduardo LM, “A review on techniques for tremor recording and quantification”, Critical Review in Biomedical Engineering, 35 (5), 2007 discloses classifying tremor as either: physiological tremor (which is present in all healthy people) or pathological tremor (associated with various diseases or conditions such as Parkinson disease). Mansur et al. concluded that physiological tremor has most energy in the [7-12] Hz domain, while pathological tremor has many components in lower ranges.
In attempting to distinguish between physiological tremor, essential tremor and Parkinsonian tremor, Jakubowski, Jacek, Kwiatos, Krzystof, Chwaleba, Augustyn, and Osowski, Stanislaw “Higher order statistics and neural network for tremor recognition”, Biomedical Engineering, IEEE Transactions on, 49 (2): 152-159, 2002 relies on a multi-layer perceptron classification of features derived from high order statistics, while Soran, Bilge, Hwang, Jenq-Neng, Lee, Su-In, and Shapiro, Linda, “Tremor detection using motion filtering and SVM”, In Pattern Recognition (ICPR), 2012 21st International Conference on, pp. 178-181, 2012 feeds a filter output to a Support Vector Machine (SVM).
Tracey K. M. Lee, Sharon S. W. Gan, J. G. Lim, Saeid Sanei, “A Multivariate Singular Spectrum Analysis Approach to Clinically-Motivated Movement Biometrics”, Eusipco 2014, 1397-1401 discloses combining two types of sensors, namely accelerometers and force sensors, in studying tremor associated with neurological disorders.
In parallel, due to their inclusion in smartphones, a multitude of applications are based on inertial sensors:
For example, Šindelář, Ondřej and Šroubek, Filip, “Image deblurring in smartphone devices using built-in inertial measurement sensors”, Journal of Electronic Imaging, 22 (1): 011003-011003, 2013 discloses removing camera shake without hardware stabilization.
Siirtola, Pekka and Röning, Juha, “Recognizing human activities user-independently on smartphones based on accelerometer data”, International Journal of Interactive Multimedia and Artificial Intelligence, 1 (5), 2012 is aimed at user activity recognition.
David Crouse, Hu Han, Deepak Chandra, Brandon Barbello, Anil Jain, “Continuous Authentication of Mobile User: Fusion of Face Image and Inertial Measurement Unit Data”, International Conference on Biometrics 2015 discloses using the accelerometer and gyroscope for determining the correct rotation of the camera and this information is used to rotate the model for the face recognition.
Yan Li, Yingjiu Li, Qiang Yan, Hancong Kong, Robert H. Deng, “Seeing Your Face Is Not Enough: An Inertial Sensor-Based Liveness Detection for Face Authentication”, Proc. of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pages 1558-1569 discloses filtering IMU sensors to remove tremor. The IMU sensors are then used for measuring the consistency between device movements and head pose changes in order to authenticate a live user's face.